Convergence in Variance of First-order and Second-order Chebyshev Accelerated Gibbs Samplers

نویسنده

  • COLIN FOX
چکیده

A stochastic version of a stationary linear iterative solver may be designed to converge in distribution to a probability distribution with a specified mean μ and covariance matrix A−1. A common example is Gibbs sampling applied to a multivariate Gaussian distribution which is a stochastic version of the Gauss-Seidel linear solver. The iteration operator that acts on the error in mean and covariance in the stochastic iteration is the same iteration operator that acts on the solution error in the linear solver, and thus both the stationary sampler and the stationary solver have the same error polynomial and geometric convergence rate. The polynomial acceleration techniques that are well-known in numerical analysis for accelerating the linear solver may also be used to accelerate the stochastic iteration. We derive first-order and second-order Chebyshev polynomial acceleration for the stochastic iteration to accelerate convergence in the mean and covariance, by mimicking the derivation for the linear solver. In particular, we show that the error polynomials are identical and hence so are the convergence rates. Thus, optimality of the Chebyshev accelerated solver implies optimality of the Chebyshev accelerated sampler. We give an algorithm for the stochastic version of the Chebyshev-accelerated SSOR (symmetric successive over-relaxation) iteration, and provide numerical examples of sampling from multivariate Gaussian distributions to confirm that the desired convergence properties are achieved in finite precision.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

A modification of Chebyshev-Halley method free from second derivatives for nonlinear equations

‎In this paper‎, ‎we present a new modification of Chebyshev-Halley‎ ‎method‎, ‎free from second derivatives‎, ‎to solve nonlinear equations‎. ‎The convergence analysis shows that our modification is third-order‎ ‎convergent‎. ‎Every iteration of this method requires one function and‎ ‎two first derivative evaluations‎. ‎So‎, ‎its efficiency index is‎ ‎$3^{1/3}=1.442$ that is better than that o...

متن کامل

An ‎E‎ffective Numerical Technique for Solving Second Order Linear Two-Point Boundary Value Problems with Deviating Argument

Based on reproducing kernel theory, an effective numerical technique is proposed for solving second order linear two-point boundary value problems with deviating argument. In this method, reproducing kernels with Chebyshev polynomial form are used (C-RKM). The convergence and an error estimation of the method are discussed. The efficiency and the accuracy of the method is demonstrated on some n...

متن کامل

Numerical Solution of a Free Boundary Problem from Heat Transfer by the Second Kind Chebyshev Wavelets

In this paper we reduce a free boundary problem from heat transfer to a weakly Singular Volterra  integral equation of the first kind. Since the first kind integral equation is ill posed, and an appropriate method for such ill posed problems is based on wavelets, then we apply the Chebyshev wavelets to solve the integral equation. Numerical implementation of the method is illustrated by two ben...

متن کامل

Partially Collapsed Gibbs Samplers: Illustrations and Applications

Among the computationally intensive methods for fitting complex multilevel models, the Gibbs sampler is especially popular owing to its simplicity and power to effectively generate samples from a high-dimensional probability distribution. The Gibbs sampler, however, is often justifiably criticized for its sometimes slow convergence, especially when it is used to fit highly structured complex mo...

متن کامل

Solving high-order partial differential equations in unbounded domains by means of double exponential second kind Chebyshev approximation

In this paper, a collocation method for solving high-order linear partial differential equations (PDEs) with variable coefficients under more general form of conditions is presented. This method is based on the approximation of the truncated double exponential second kind Chebyshev (ESC) series. The definition of the partial derivative is presented and derived as new operational matrices of der...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2012